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Abstract — In this paper, we consider a few iterative decoding 
schemes for the joint source-channel coding of correlated 
sources. Specifically, we consider the joint source-channel cod- 
ing of two erasure correlated sources with transmission over 
different erasure channels. Our main interest is in determining 
whether or not various code ensembles can achieve the capacity 
region universally over varying channel conditions. We consider 
two ensembles in the class of low-density generator-matrix 
(LDGM) codes known as Luby-Transform (LT) codes and one 
ensemble of low-density parity-check (LDPC) codes. We analyze 
them using density evolution and show that optimized LT codes 
can achieve the extremal symmetric point of the capacity region. 
We also show that LT codes are not universal under iterative 
decoding for this problem because they cannot simultaneously 
achieve the extremal symmetric point and a corner point of the 
capacity region. The sub-universality of iterative decoding is 
characterized by studying the density evolution for LT codes. 

I. Introduction 

The system model considered in this paper is shown 
in Figure |l(a)| We wish to transmit the outputs of two 
discrete memoryless correlated sources UJ^\u^J, for 
i = 1, 2, • • • , fc to a central receiver through two independent 
discrete memoryless channels with capacities C\ and C2, 
respectively. We will assume that each channel can be param- 
eterized by a single parameter for i = 1, 2 (e.g., the erasure 
probability or crossover probability). The two sources are not 
allowed to collaborate and, hence, they use two independent 
encoding functions which map the k input symbols in to 
n\ and 122 output symbols, respectively. The rates of the 
encoders are given by i?i = kjn\ and R2 = k/ri2- 

In such a problem, it is clear that one has to take advantage 
of the correlation between the sources to reduce the required 
bandwidth to transmit the information to the central receiver. 
Thus, this joint source-channel coding problem can be seen 
to be an instance of Slepian-Wolf coding [1] in the presence 
of a noisy channel. If ei and £2 are known to transmitter 1 
and 2 respectively, then the sources can be reliably decoded 
at the receiver iff 
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In this case, one can separate the problem into Slepian-Wolf 
coding [1] of the two sources and channel coding for the 
two channels. In recent years, there have been graph based 
coding schemes which, under iterative decoding, can obtain 
near optimal performance for this problem [2], [3], [4], [5]. 

However, in several practical situations, it is unrealistic 
for the transmitters to have a priori knowledge of t\ and £2. 
Therefore, we consider the case where the transmitters each 
use a single code of rate R (though it is possible to extend 
this to different rates R\ and R^). We then wish to find a 
universal source-channel coding scheme such that reliable 
transmission is possible over a range of channel parameters 
(£1,62). Ideally, we would like to have one code of rate 
R\ = R'2 = R that allows error free communication of 
the sources for any set of channel parameters (£i,£2) for 
which £1, £2 satisfy the conditions in £[). For a given pair of 
encoding functions of rate R and a joint decoding algorithm, 
the achievable channel parameter region (ACPR) is defined 
as the set of all channel parameters (£i,£2) for which 
the encoder/decoder combination achieves an arbitrarily low 
probability of error as k — ► 00. For some channels, this 
region is equal to the entire region in (H) and, in this case, 
we call it the capacity region. Note that the ACPR and 
the capacity region are defined as the set of all channel 
parameters for which successful recovery of the sources is 



possible for a fixed encoding rate pair (R, R) (or, more 
generally (RijRz)) instead of the set of rates (i?i,i?2) for 
a fixed pair of channel parameters (ei, £2). 

It can be seen that the capacity region is, in fact, given by 
all pairs of (e l5 e 2 ) such that (fl]i is satisfied. For binary-input 
memoryless symmetric channels, this region is achieved 
when both users encode with independent random linear 
codes and use maximum-likelihood (or typical set) decoding 
at the receiver. This means that random codes with ML 
decoding are universal for symmetric channels. That is, for 
a given (i?i,i?2), a single encoder/decoder pair suffices 
to communicate the sources over all pairs of symmetric 
channels for which (£1,62) satisfy the conditions in (Q]). 
Thus, one can obtain optimal performance even without 
knowledge of (£1,62) at the transmitter. We refer to such 
encoder/decoder pairs as being universal. 

While random codes with ML decoding are universally 
good, this scheme is clearly impractical due to its large 
complexity. Our primary interest in this paper is to investigate 
whether there exist graph based codes and iterative decoding 
algorithms that are also universal and to find good en- 
coder/decoder pairs that result in large ACPRs. Several code 
ensembles, including Luby Transform (LT) codes and LDPC 
codes, have been shown to achieve capacity with iterative 
decoding on a single user erasure channel [6], [7]. However, 
the universality of these ensembles for more complicated 
scenarios has not been studied well in the literature. Hence, 
the question of whether one can design a single graph 
based code and a decoding algorithm capable of universal 
performance is a question that has not been answered in the 
literature. One of the main results in this paper is that iterative 
decoding of LT codes cannot be universal thus showing that 
ensembles that are good for single user channels do not 
necessarily perform well for the joint source-channel coding 
problem. 

Before we discuss the main results in this paper in 
Section Hill we first introduce a specific instance of the 
problem described above which is simple and yet captures 
the difficulty of designing a universal joint source-channel 
coding scheme. 

II. System Model for Erasure Correlation 

Consider the case where the source correlation and chan- 
nels both have an erasure structure. Let Zi, for i — 
1, 2, • • • , k, denoted by the column vector Zfc, be a sequence 
of i.i.d. Bernoulli-p random variables. The correlation be- 
tween U^ 1 ** and U' 2 ' is defined by 

! i.i.d. Bernoulli i r.v.s, if Zi = 
same Bernoulli ^ r.v. Ui ,if Z^ = 1 

We consider transmission over erasure channels with erasure 
rates t\ and £2- The Slepian-Wolf conditions are satisfied if 

(l-£l) > (i-p)-Ri 
(l-£ 2 ) > {l- P )R 2 
(1-ei) , (1-ea) 



and the achievable channel parameter region is shown in Fig. 

El 

The source sequences and U^ 2 ) are encoded using a 
pair of independent binary linear codes C\ [n, k) and C2[n, k] 
chosen from the same code ensemble. We consider the 
encoding and decoding of LDPC codes and LDGM codes 
separately. 

A. LDGM codes 

The source sequences are encoded using different LDGM 
codes chosen from the same ensemble, defined in terms of 
generator matrices G (1) and G {2 \ The encoded sequences 
denoted by and X^ 2 ) are given by 



The source bits U^ 1 -* and U^ 2 ) are punctured and then 
transmitted through binary erasure channels (BECs) with 
erasure rates e\ and £2 respectively. The governing equations 
at the decoder are given by 



In 



X (i) = 0, for i = 1,2. 



where /„ is an n x n identity matrix. For simplicity of 



for i = 1,2, for the 



notation, we define ffW = G^ T I n 
case of LDGM codes. Given a matrix A, and a suitable index 
set I, let Ax (Ax>) denote the sub-matrix of A, restricted to 
the columns (rows) indexed by X. Let V denote the set of 
indices corresponding to the non-zero locations of Zfe, and Z 
be the diagonal matrix, whose diagonal is given by [Zfe,0], 
where denotes a vector of all zeros of appropriate length. 
The governing equation i/X = at the joint decoder can 
therefore be written in terms of the stacked parity check 
matrix 

" 



Z-p' Z'pr 



H = 
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where X = [XW, X^ 2 '] and [• , •] denotes concatenation. 

B. LDPC codes 

The source sequences are encoded using LDPC codes, 
defined in terms of parity-check matrices iJW and H ^ . The 
encoded sequences, denoted X^ and X^ 2 ), are encoded us- 
ing a punctured systematic encoder and transmitted through 
binary erasure channels (BECs) with erasure rates t\ and 
£2 respectively. The governing equations at the decoder are 
given by 

ff( l) X (l) = 0, for i = 1,2. 

For joint decoding, the governing equations (including the 
source correlation constraints), written in terms of the 
stacked parity check matrix defined in (0, are given by 



HX = 0, 



R2 



>2-p, 



where X= [X«, X( 2 )]. 



C. Maximum Likelihood Block Decoder 



IV. Design and Analysis of LDGM Codes 



In the case of an erasure channel, ML decoding of linear 
codes is equivalent to solving systems of linear equations, 
which can be performed using Gaussian elimination. Let £\, 
£2 (and £\,£<A denote the index sets of erasures (and non- 
erasures) corresponding to the received vectors, and let £ = 
[Si, k + n + £ 2] , £ — [£1, k + n + £2] ■ Denote the received 
sequences by and Y^ 2 ) with Y = [YW, Y^ 2 )]. For 
the binary case, the defining equation HY = simplifies 
to HgYg = HgYg, in this case. Block ML decoding will 
be successful iff Hg has full rank and the erasures can be 
recovered by inverting Hg. 

D. Example 

For example, consider the case where k — 4 and n — 3 
using the LDPC framework. Then, we can choose C\ and C2 
be [7,4] Hamming codes. If Z 4 = [l l] T , then the 
stacked parity-check matrix is given by 



A. Density Evolution Equations 
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The Tanner graph corresponding to the stacked parity check 
matrix for LDGM codes is shown in Fig. [2] and iterative 
decoding is performed on this Tanner graph. 

III. Outline of the paper and summary of results 

We now summarize the main results of this paper. 

• In Section [IV] we consider the design and analysis of 
LDGM codes. We derive the density evolution equations 
for LT codes in Section IIV-AI In Section IIV-BI we 
consider the design of LDGM codes for the extremal 
symmetric point and the corner point of the capacity 
region using linear programming. 

• In Section IIV-CI we first show analytically that LT 
codes with iterative decoding can achieve the extremal 
symmetric point of the ACPR. However, they cannot 
achieve the corner point of the ACPR and, hence, LT 
codes with iterative decoding cannot be universal for 
the joint source-channel coding problem. 

• In Section [V] we show from simulations that LT codes 
and the (4,6) LDPC code using maximum likelihood 
decoding are nearly universal. 

These results essentially show that the problem in obtaining 
universality with the LT ensemble is essentially with the 
decoding algorithm rather than with code ensemble. This 
motivates us to find other decoding algorithms such as 
enhancements to message passing decoding that are nearly 
universal or to consider other code ensembles than the LT 
code ensemble with iterative decoding. 



Assume that the sequences £/W and are encoded 

using LT codes with degree distribution pairs (A« p«),for 
2 = 1,2. Based on standard notation [7], for i = 1, 2, we let 
X^(x) = J2j ^f 1 x3 ~ 1 b e tne degree distribution (from an 
edge perspective) corresponding to the information variable 
nodes and p^{x) = J2j pf xi~ l be the degree distribution 
(from an edge perspective) of the generator (aka check) 
nodes in the decoding graph. The coefficient \y (resp. py) 
gives the fraction of edges that connect to the information 
variable nodes (resp. generator nodes) of degree j. Likewise, 
L {i) (x) = Y,j L j l)xj ( res P- R [l) {x) = Hj R f x °) is the 
degree distributions from the node perspective and Lj (resp. 

Rj^) is the fraction of information variable (resp. generator) 
nodes with degree j. 

Since the encoded variable nodes are are attached to 
generator nodes randomly, the degree of a each information 
variable is a Poisson random variable whose mean is given by 
the average number of edges attached to each variable node. 
This mean is given by a; = R'^ l \l)/Ri, where R'^(l) 
is the average generator (or check) degree. Therefore, the 
resulting degree distribution is L^'{x) = e Qi ' 1-1 ' for i — 
1,2. 

The Tanner graph [7] for the code is shown in Fig. [2] 
from which the density evolution equations [7] in terms 
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Fig. 2. Tanner Graph of an LT Code with erasure correlation between the 
sources 



of the generator-node to variable-node messages (a;, and 
corresponding to codes 1 and 2) can be written as follows 

Xi+i = 1 - (1 - (l - ((1 -p) +pL^( Vl )) A^)) 

y l+ i = 1 - (1 - £ 2 )P (2) (l - ((1 -p) + P L^( Xi )) \^(y t )) , 

where L^(x), for i = 1,2, are the degree distributions (from 
the node perspective) corresponding to the information bits. 
For analysis, it is easier to consider the evolution of the 



Xi+1 = 

Vi+i = 



variable-node to generator-node messages, given by 

a (1) (e^i.xO) 
a (2) (e^W)). 

where £> W (e,x) = 1 - (1 - e)pW(l - x). Notice that, for LT 
codes, the variable-node degree distribution from the edge 
perspective is given by AW(x) = lW(s) because A(x) = 
L'(x)/L'(\) for Poisson L(x). With this simplification, the 
density evolution equations can be written as 



The solution is given by 



w+i — 



A (1) (^(ex.x,)) 
A (2) fflW( Ca , W )' 



B. Optimization of degree distributions via Linear Program- 
ming 

We use linear programming to design two LT codes. The 
first code, called LT code I, is designed using the successful 
decoding constraints for the extremal symmetric point, given 
by the channel condition (e, e) = (l — ^^-R, 1 — 2 -^ £ i?), as 
follows. 

• Choose the maximum check degree to be AT. 

. Compute a = 1+ f_ w £ (p) , with G N (p) as defined in ©. 

• Maximize J^i Pi/h subject to, V x <E [0, 1], 



^2 Pi ■ (1 - ((1 ~P) +P K (c,x)) K(e,x)Y 1 < x, 

l<i<N 

(3) 

where /c(e,x) = e * 1 "')^- 1 ). 
The constraints in (0) are obtained from the density evolution 
equations, in terms of the generator-node to variable-node 
messages, described in Section IIV-CI (the messages corre- 
spond to a modified Tanner graph, where all the generator 
nodes corresponding to the erasures in the received sequence 
have been removed). To achieve a corner point in the Slepian- 
Wolf region, given by the channel condition (£1,62) = 
(1 — (1 — p)R, 1 — R), the constraints in (01 were added 
(obtained from the density evolution equations described in 
IIV-DI assuming that the code corresponding to the better 
channel has converged). This gives, Vie [0, 1], 

p t -(l-((l-p)+P^ei,0))K(e 2 ,x)y- 1 <x. (4) 

l<i<JV 

C. The extremal symmetric point 

We first analyze a code optimized for the case when both 
channels have the same erasure probability (ei = e 2 = e), 
to understand the criteria for achieving universality. Due to 
the symmetry of the model for this case, we have pW(x) = 
p W(x) = p(x) and \W(x) = \ {2) (x) = X(x) = e^ 1 " 1 ), 
and the density evolution equations collapse into a one- 
dimensional recursion, given by 

x m =[(l-p)+pA(l-(l-e)p(l-x i ))]A(l-(l-e)p(l-x i )). 

This recursion can be solved analytically, resulting in the 
unique non-negative p(x) which satisfies 

x = [(l-p) +p A(l-(l-e)p(l-x))]A(l-(l-e)p(l-x)). 



p(x) 
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a(l-e) 



log 
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1 ^n-j (V)p k . 



a(l-e)^ ifl+p) 2 *- 1 

and we note that is not a valid degree distribution because it 
has infinite mean. To overcome this, we define a truncated 
version of the generator degree distribution via 



p N (x) 



1 2~di—\ i(l+p)2i-l \ X 



p + G N (p) + 1 



G N (p) = ^ 



N El-Jo ( 2 TV 



(5) 



-{ »(1+P) 
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for some p > and N G N. This is a well defined 
degree distribution as all the coefficients are non-negative and 
p N (l) = 1. The parameter p increases the number of degree 
one generator nodes and is introduced in order to overcome 
the stability problem at the beginning of the decoding process 
[6]. 

Theorem 4.1: Consider transmission over erasure chan- 
nels with parameters t\ = e 2 = e. Let N E N and p > 
and 

P + G N ( P ) + 1 



where 



Gn(p) 



1 



\2i-l 



Then, in the limit of infinite blocklengths, the ensemble 
LDGM(n, X(x),p N (x)), where 



A(x) ^e"^" 1 ', 



p N (x) 



k=o ( k )P i , N 

P^ 1^,1=1 i(l+p)2*-l X ^X 



(6) 



p + G N (p) + 1 



enables transmission at a rate R = ^ ^f 1 e ,„ s with a bit 



error probability not exceeding 1/N. 

Proof: See Appendix U ■ 
From Theorem 14.11 we conclude that optimized LT codes, 
given by the ensemble LDGM(n, A(x), p (x)J can achieve 
the extremal symmetric point of the capacity region. 

D. A Corner Point 

Consider the performance of the ensemble 
LDGM(n, A(x),pW(x)), with A(x) and pW(x) as 
defined in ©, at a corner point of the Slepian-Wolf 
region. One corner point is given by the channel condition 
( Cl ,e 2 ) = (1 - (1 -p)R,l - (1 -p/2)R). The density 
evolution equations are 

Xj+i = [(1 -p) +pX N (e 2 ,yi)] X N (e!,Xi) 



[(l-p)+pX N (e 1 ,x i )] X N (e 2 , yi ) 



(7) 



where X N (e,x) = A (l - (1 - e)p N (l-x)). 

Theorem 4.2: LT codes cannot simultaneously achieve the 
extremal symmetric point and a corner point of the Slepian- 
Wolf region, under iterative decoding. 

Proof: See Appendix HI1 ■ 
From Theorem 14.21 we conclude that LT codes designed for 
the extremal symmetric point are not universal for the two- 
user Slepian-Wolf problem, with erasure correlated sources. 

V. Performance of Various Code Ensembles 

In this section, we study the performance of three code 
ensembles under iterative and maximum likelihood decoding 
using simulations. The codes considered are 

1 ) A linear code with a random generator matrix. 

2) A (4, 6) regular LDPC code with punctured systematic 
bits. 

3) Two LT codes (LT code I and LT code II) optimized 
for different points in the capacity region. 

LT code I is optimized for the case when both channels have 
the same erasure probability (i.e., the extremal symmetric 
point of the capacity region). LT code II is optimized for 
the extremal symmetric point, including constraints corre- 
sponding to channel conditions at one corner of the capacity 
region. Joint iterative decoding is performed on the Tanner 
graph corresponding to the stacked parity check matrix H. 
The simplified message passing rules for the BEC are used. 
They are stated here for convenience. At a variable node, 
the outgoing message is an erasure if all incoming messages 
are erasures. Otherwise, all non-erasure messages must have 
the same value, and the outgoing message is equal to the 
common value. At the check node, the outgoing message is 
an erasure if any of the incoming messages is an erasure. 
Otherwise, the outgoing message is the XOR of all the 
incoming messages. Joint ML decoding was performed on 
the stacked parity check matrix as described in Section ITl-AI 
The simulations were performed with codes of rate 1/2 
(i.e., two encoded bits are generated per source bit), and a 
blocklength of 500. We chose a source correlation of p = 
0.5, and simulated 300 blocks for each point in the capacity 
region. All the plots are shown in the (ei, e2)-plane, for the 
rate pair (1/2, 1/2). 

A. Random Codes 

Two different codes of rate 1/2 are chosen randomly 
from the generator-matrix ensemble, where the entries of the 
generator matrix are i.i.d. Bernoulli-1/2 random variables. 
Decoding was performed on the stacked parity-check matrix 
corresponding to LDGM codes. The ACPR of random codes 
under iterative and ML decoding is shown in Fig. IV-AI 
respectively. As expected, random codes achieve the entire 
capacity region under ML decoding, but perform very poorly 
under iterative decoding. The ACPR with iterative decoding 
consists of only 3 non-trivial points with channel parameters 
very close to zero. 
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Fig. 3. ACPR for a Random Code under ML Decoding 

B. LT Codes 

LT codes have been shown to be universal for the single- 
user erasure channel. Here, we study the performance of 
LT codes for the two-user erasure channel and consider the 
case of encoding and decoding at the extremal symmetric 
point of the Slepian-Wolf region. An LT code is optimized 
for this point (LT Code I), using linear programming (see 
Section |IV-B1 >, resulting in the degree distribution given by 

p(x) = 0.0001 + 0.0754 • x + 0.0295 • x 2 + 0.0620 • x 3 + 
0.0857 • x 7 + 0.0718 • x 15 + 0.0970 • x 31 + 
0.0114- a; 63 + 0.5671 • x 127 . 

The performance of this code under iterative decoding is 
shown in Fig. |4(a)| Also shown in Fig. |4(a)| is the simulated 
density evolution threshold for LT code I. The density 
evolution threshold at the extremal symmetric point is away 
from capacity for this code due to limiting the maximum 
check degree in the design process. Also, note that the 
density evolution threshold is far away from capacity at the 
corner points of the capacity region. On the other hand, as 
seen in Fig. |4(b)| the code performs much better under ML 
decoding and is closer to capacity at the corner points of 
the capacity region. This reinforces the conclusion that most 
of the sub-universality is due to the iterative decoder, rather 
than the stability of the code. 

In order to achieve capacity at the corner points of the 
capacity region, LT code II was designed by adding con- 
straints corresponding to the channel conditions at a corner 
point of the capacity region (see Section |IV-B| i, resulting in 
the following degree distribution, 

p(x) = 0.0001 + 0.0640 • x + 0.0251 ■ x 2 + 0.0526 • x 3 + 
0.0725 • x 7 + 0.0619 • x 15 + 0.0806 • x 31 + 
0.0082 -a; 63 + 0.6351 -a; 127 . 

The performance of this code under iterative decoding (and 
the simulated density evolution threshold) ML decoding is 
shown in Fig. |5(a)| and Fig. |5(b)| respectively. Note that the 
density evolution threshold increases only marginally, and 
the performance under ML decoding is almost the same. 



C. LDPC Codes 

Here, we consider the performance of a punctured (4, 6) 
LDPC code for the joint source-channel coding problem. 
Two systematic codes were chosen from the ensemble 
LDPC(4, 6), and the systematic bits are punctured before 
transmission, resulting in a code of rate 1/2 (two encoded 
bits are transmitted per source bit). 

The (4, 6) codes achieve the entire capacity region under 
ML decoding, as shown in Fig. |6(a)| and the iterative 
decoding threshold is significantly lower as seen in Fig. |6(b)| 
Again, this shows that the iterative decoder is the main reason 
for the loss of universality. 

VI. Conclusions and Future Work 

In this paper, we considered the performance of graph 
based codes with iterative decoding for obtaining univer- 
sal performance when transmitting correlated sources over 
binary erasure channels. We designed an LT code which 
can achieve the extremal symmetric point. We then showed 
that an LT code optimized for the symmetric sum-rate point 
cannot achieve a corner point of the capacity region and, 
hence, we concluded that LT codes cannot be universal 
for this two user Slepian-Wolf problem. Our simulation 
results indicate that a punctured LDPC code ensemble and 
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(b) ACPR for LT Code I under ML Decoding 
Fig. 4. Performance of LT Code I 
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(a) ACPR for LT Code II under Iterative Decoding 
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(b) ACPR for LT Code II under ML Decoding 
Fig. 5. Performance of LT Code II 



LT ensemble are nearly universal with maximum likelihood 
decoding. 

For future work, we plan to do the following. 

• Analyze the performance of a carefully designed proto- 
graph code to try and achieve universality with iterative 
decoding. 

• Since ML decoding is nearly universal and iterative 
decoding is not universal, we would like to see if there is 
an enhancement to iterative decoding that can be nearly 
universal but is yet significantly less complex than ML 
decoding. 

Appendix I 
Proof of TheoremI4.1I 



We will use the following Lemma to show that the 
density evolution equations converge to zero at the extremal 
symmetric point. 

Lemma 1.1: 

P N (x)> - , for 0< *<!-!. 
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(b) ACPR for a (4, 6) LDPC Code under ML Decoding 
Fig. 6. Performance of (4, 6) LDPC Codes 

Proof: For Q < x < 1 — jr, we have 



p N (x) = 



/2i-l\ k 

N \ k=0 ( fc IP „j 
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JV 



M + G w (p) + 1 
fi + p(x) + x N 



(i + G N (p) + l 



i=iV+l i(l+p) 2 
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[i + G N { P ) + l 
p + p(x) 



p + G N (p) + l 
The last step follows from the fact that 
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i=JV+l 
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iV 
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l=iV+l 



iV+ 1 1 - a; 



where the last step follows from explicit calculations, taking 
into account that < a; < 1 — -4. ■ 
From (IIV-CI ). the convergence criteria for the density evolu- 
tion equation is given by 

x>[(l-p)+ P X N (e,x)] X N (e,x) 

Consider the term X N (e, x) = A (l - (1 - e)p N (l - x)). 
We have, 

X N (e,x) = e -a(i-0-^(i-x) 

nfl rl >'+p( 1 ~ x ) 1 
<e U ^m+GjvCpJ + i if j; > _ 

- n 



< e 



^/(l-p) 2 + 4px- (1-p) 
2p 



< 



v /(l-p) 2 +4px-(l-p) 
2p 



where the first inequality follows from Lemma 11.11 The 
polynomial f(y) = py 2 + (1 — p)y — x is a convex function 

^(l-p) 2 +4 pa: -(l-p) 
2p 



of y, with the only positive root at y 



So, if y < Vd-ri'+^-d- p), then < o. Hence, 

[(1 — p) +pA(e, ar)l A(e, x) — x < and the density evo- 
lution equation converges, as long as i > -h. So, the 
probability of erasure is upper bounded by 1 /N. 
The rate of the code is computed as 



R 



X(x) dx 
J^pW(x)dx 



We have 



also 



lim 

AT-KX 



P N (x)dx 



o N {x)dx 



i(l + l)(l+p) 21 - 1- 



iV+1 



/i + Gat (p) + 1 



p(a;) da; 



2 



Note that p( N ^ (x) dx is a monotonically increasing se- 
quence, upper bounded by 1 — |. So, in the limit of infinite 
blocklengths the design rate is given by 

(l-e)(l-e-«) 



i? = 



Appendix II 
Proof of Theorem I4.2I 

To analyze the convergence of the ensemble 
LDGM(n, X(x) : p( N ^(x)), consider the functions 

f(x,y) = [(1 -p) +p~X N (e 2: y)] X N (e 1: x) - x 
g(x,y)=[(l-p)+ P X N (e 1 ,x)] X N (e 2 ,y)-y. 

The condition for convergence of the density evolution 
equations are given by f[x,y) < and g{x,y) < 0. When 



ei < e 2 , we can approximately characterize the convergence 
by analyzing the condition g(0,y) < 0. We have 

.9(0, y) = [(1 -p) +p\(e 1 )} A (l - (1 - e 2 )p N (l - y)) - y 
< [(1 - p) +pA(ei)] A (1 - (1 - e 2 )p{\ - y)) - y 

= k (yi + ay-l) -y, 



where 

k 



2p 



(1 -p) +pe" 



«(l-ei) 



1-62 



and a 



4p 



l-e ~ " :1 V? 
The fixed point of g(0, y) can be found by solving 



i.e., 



y = fe (yi + ay- 1 
This equation is of the form 

k -2/0 y( 2,P-l) + 2k -l/ Pyl l/ P -l) _ a = Qj 

the root of which is approximately equal to the root of the 
quadratic 

k- 2 'Pz 2 + 2k-Wz - a, 



where z — y^ 1 / 13 1 / 2 " 1 
is given by z ■ 



evolution is y 

[(1-p) 



Th e positive root of the quadratic 

2/3 
2-/3 



fe-V/3 



( 2p \ 

\(l-p)k- 1 /l : > J 



2(l-p) 



2p 

e ^ LU _p ;+ pe-°(i-*i)]j 2(1 - p) >0. 
Due to the presence of a constant fixed point, which does 
not approach even in the limit of infinite maximum degree, 
the residual erasure rate is always bounded away from 0. So, 
the ensemble LDGM(n, X(x), p( N > (x)) cannot converge at 
a corner point of the capacity region. 
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